从语言模型到大型语言模型

From Language Models to Large Language Models

虽然语言模型已经存在一段时间了，但它们只有在自监督的框架下才能发展到如今的规模。本节将简要概述语言模型和自监督的含义。如果您已经熟悉这些概念，请跳过本节。

While language models have been around for a while, they’ve only been able to grow to the scale they are today with self-supervision. This section gives a quick overview of what language model and self-supervision mean. If you’re already familiar with those, feel free to skip this section.

语言模型

Language models

语言模型会编码一种或多种语言的统计信息。直观地说，这些信息告诉我们一个单词在特定语境中出现的可能性。例如，给定语境“我最喜欢的颜色是__”，编码英语的语言模型应该会更频繁地预测“蓝色”而不是“汽车”。

A language model encodes statistical information about one or more languages. Intuitively, this information tells us how likely a word is to appear in a given context. For example, given the context “My favorite color is __”, a language model that encodes English should predict “blue” more often than “car”.

语言的统计特性早在几个世纪前就已被发现。在1905年的小说《跳舞的人》中，夏洛克·福尔摩斯利用英语的简单统计信息，破译了一串神秘的火柴人图案。由于英语中最常用的字母是E，福尔摩斯推断，最常见的火柴人图案一定代表E。

The statistical nature of languages was discovered centuries ago. In the 1905 story “The Adventure of the Dancing Men”, Sherlock Holmes leveraged simple statistical information of English to decode sequences of mysterious stick figures. Since the most common letter in English is E, Holmes deduced that the most common stick figure must stand for E.

后来，克劳德·香农在第二次世界大战期间运用更为复杂的统计学方法破译了敌人的信息。他关于如何对英语进行建模的研究发表于1951年的里程碑式论文《印刷英语的预测与熵》。这篇论文中提出的许多概念，包括熵，至今仍在语言建模中得到应用。

Later on, Claude Shannon used more sophisticated statistics to decipher enemies’ messages during the Second World War. His work on how to model English was published in his 1951 landmark paper “Prediction and Entropy of Printed English”. Many concepts introduced in this paper, including entropy, are still used for language modeling today.

早期一个语言模型只涉及一种语言，而今天一个语言模型可以涉及多种语言。

In the early days, a language model involved one language. However, today, a language model can involve multiple languages.

语言模型的基本单位是token。一个 token 可以是一个字符、一个单词或一个单词的一部分（如 -tion），具体取决于模型。2^例如，ChatGPT 背后的模型 GPT-4 将短语“I can't wait to build AI applications”分解为 9 个 token，如图 1-1所示。请注意，在此示例中，“can't”这个词被分解为两个 token，分别是 can和't 。您可以在OpenAI 网站上查看不同的 OpenAI 模型如何对文本进行标记。

The basic unit of a language model is token. A token can be a character, a word, or a part of a word (like -tion), depending on the model.² For example, GPT-4, a model behind ChatGPT, breaks the phrase “I can’t wait to build AI applications” into nine tokens, as shown in Figure 1-1. Note that in this example, the word “can’t” is broken into two tokens, can and ’t. You can see how different OpenAI models tokenize text on the OpenAI website.

图 1-1。GPT -4 如何标记短语的示例。

将原文分解成标记的过程称为标记化。对于 GPT-4 来说，一个标记的平均长度约为一个单词的 ¾ 。因此，100 个标记大约相当于 75 个单词。

The process of breaking the original text into tokens is called tokenization. For GPT-4, an average token is approximately ¾ the length of a word. So, 100 tokens are approximately 75 words.

模型可以使用的所有标记的集合称为模型的词汇表。您可以使用少量标记来构造大量不同的单词，就像使用字母表中的几个字母来构造许多单词一样。Mixtral 8x7B模型的词汇量为 32,000。GPT-4 的词汇量为100,256。标记化方法和词汇量由模型开发者决定。

The set of all tokens a model can work with is the model’s vocabulary. You can use a small number of tokens to construct a large number of distinct words, similar to how you can use a few letters in the alphabet to construct many words. The Mixtral 8x7B model has a vocabulary size of 32,000. GPT-4’s vocabulary size is 100,256. The tokenization method and vocabulary size are decided by model developers.

笔记

为什么语言模型使用token作为单位，而不是使用word或character呢？主要有三个原因：

Why do language models use token as their unit instead of word or character? There are three main reasons:

1. 与字符相比，token 允许模型将单词分解成有意义的部分。例如，“cooking” 可以分解成“cook”和“ing”，这两个部分都包含原词的一些含义。

2. Compared to characters, tokens allow the model to break words into meaningful components. For example, “cooking” can be broken into “cook” and “ing”, with both components carrying some meaning of the original word.

3. 由于唯一标记的数量比唯一单词的数量少，这减少了模型的词汇量，从而使模型更高效（如第 2 章所述）。

4. Because there are fewer unique tokens than unique words, this reduces the model’s vocabulary size, making the model more efficient (as discussed in Chapter 2).

5. 标记还能帮助模型处理未知单词。例如，像“chatgpting”这样的生造词可以拆分成“chatgpt”和“ing”，帮助模型理解其结构。标记在保留比单词更少的单元和比单个字符更多含义之间取得了平衡。

6. Tokens also help the model process unknown words. For instance, a made-up word like “chatgpting” could be split into “chatgpt” and “ing”, helping the model understand its structure. Tokens balance having fewer units than words while retaining more meaning than individual characters.

语言模型主要有两种类型：掩码语言模型和自回归语言模型。它们的区别在于可以用来预测标记的信息：

There are two main types of language models: masked language models and autoregressive language models. They differ based on what information they can use to predict a token:

掩蔽语言模型

Masked language model

掩码语言模型经过训练，可以使用缺失标记前后的上下文来预测序列中任何位置的缺失标记。本质上，掩码语言模型被训练成能够填补空白。例如，给定上下文“我最喜欢的__是蓝色”，掩码语言模型应该预测空白很可能是“颜色”。一个著名的掩码语言模型示例是来自 Transformer 的双向编码器表示，即 BERT（Devlin 等人，2018）。

A masked language model is trained to predict missing tokens anywhere in a sequence, using the context from both before and after the missing tokens. In essence, a masked language model is trained to be able to fill in the blank. For example, given the context, “My favorite __ is blue”, a masked language model should predict that the blank is likely “color”. A well-known example of a masked language model is bidirectional encoder representations from transformers, or BERT (Devlin et al., 2018).

截至撰写本文时，掩码语言模型通常用于非生成性任务，例如情感分析和文本分类。它们对于需要理解整体上下文的任务也很有用，例如代码调试，其中模型需要理解前后代码以识别错误。

As of writing, masked language models are commonly used for non-generative tasks such as sentiment analysis and text classification. They are also useful for tasks requiring an understanding of the overall context, such as code debugging, where a model needs to understand both the preceding and following code to identify errors.

自回归语言模型

Autoregressive language model

自回归语言模型经过训练，可以仅使用序列中的前一个标记来预测下一个标记。它预测了“我最喜欢的颜色是__”中接下来会出现什么^。3自回归模型可以连续生成一个又^一个标记。如今，自回归语言模型已成为文本生成的首选模型，因此，它们比掩码语言模型更受欢迎。4

An autoregressive language model is trained to predict the next token in a sequence, using only the preceding tokens. It predicts what comes next in “My favorite color is __.”³ An autoregressive model can continually generate one token after another. Today, autoregressive language models are the models of choice for text generation, and for this reason, they are much more popular than masked language models.⁴

图1-2展示了这两类语言模型。

Figure 1-2 shows these two types of language models.

图 1-2。自回归语言模型和掩蔽语言模型。

笔记

在本书中，除非明确说明，语言模型均指自回归模型。

In this book, unless explicitly stated, language model will refer to an autoregressive model.

语言模型的输出是开放式的。语言模型可以利用其固定的、有限的词汇量来构建无限可能的输出。能够生成开放式输出的模型被称为生成式模型，因此被称为生成式人工智能。

The outputs of language models are open-ended. A language model can use its fixed, finite vocabulary to construct infinite possible outputs. A model that can generate open-ended outputs is called generative, hence the term generative AI.

你可以把语言模型想象成一个补全机器：给定一个文本（提示），它会尝试补全该文本。以下是一个例子：

You can think of a language model as a completion machine: given a text (prompt), it tries to complete that text. Here’s an example:

提示（来自用户）：“To be or not to be”

完成（来自语言模型）：“, that is the question.”

Prompt (from user): “To be or not to be”

Completion (from language model): “, that is the question.”

需要注意的是，补全是基于概率的预测，并不保证正确。语言模型的这种概率特性使得它们的使用既令人兴奋，又令人沮丧。我们将在第二章进一步探讨这一点。

It’s important to note that completions are predictions, based on probabilities, and not guaranteed to be correct. This probabilistic nature of language models makes them both so exciting and frustrating to use. We explore this further in Chapter 2.

补全功能听起来很简单，但实际上却非常强大。许多任务，包括翻译、摘要、编程和解决数学问题，都可以定义为补全任务。例如，给出提示：“你用法语怎么说……”，语言模型可以用“Comment ça va”来补全它，从而有效地将一种语言翻译成另一种语言。

As simple as it sounds, completion is incredibly powerful. Many tasks, including translation, summarization, coding, and solving math problems, can be framed as completion tasks. For example, given the prompt: “How are you in French is …”, a language model might be able to complete it with: “Comment ça va”, effectively translating from one language to another.

再举一个例子，给出提示：

As another example, given the prompt:

Question: Is this email likely spam? Here’s the email: <email content>

Answer:

Question: Is this email likely spam? Here’s the email: <email content>

Answer:

语言模型可能能够用“可能是垃圾邮件”来完成它，这会将该语言模型变成垃圾邮件分类器。

A language model might be able to complete it with: “Likely spam”, which turns this language model into a spam classifier.

虽然补全功能很强大，但补全并不等同于参与对话。例如，如果你向补全机器提问，它可能会通过添加另一个问题而不是回答问题来补充你所说的内容。“训练后”部分讨论了如何让模型恰当地响应用户的请求。

While completion is powerful, completion isn’t the same as engaging in a conversation. For example, if you ask a completion machine a question, it can complete what you said by adding another question instead of answering the question. “Post-Training” discusses how to make a model respond appropriately to a user’s request.

从大型语言模型到基础模型

From Large Language Models to Foundation Models

虽然语言模型能够完成许多不可思议的任务，但它们仅限于文本。作为人类，我们感知世界的方式不仅包括语言，还包括视觉、听觉、触觉等等。能够处理文本以外的数据对于人工智能在现实世界中的运作至关重要。

While language models are capable of incredible tasks, they are limited to text. As humans, we perceive the world not just via language but also through vision, hearing, touch, and more. Being able to process data beyond text is essential for AI to operate in the real world.

因此，语言模型正在不断扩展，以融入更多数据模态。GPT-4V 和 Claude 3 可以理解图像和文本。有些模型甚至可以理解视频、3D 资产、蛋白质结构等等。将更多数据模态融入语言模型，使其更加强大。OpenAI在其 2023 年的 GPT-4V 系统卡中指出，“将更多模态（例如图像输入）融入法学硕士 (LLM) 被一些人视为人工智能研发的关键前沿。”

For this reason, language models are being extended to incorporate more data modalities. GPT-4V and Claude 3 can understand images and texts. Some models even understand videos, 3D assets, protein structures, and so on. Incorporating more data modalities into language models makes them even more powerful. OpenAI noted in their GPT-4V system card in 2023 that “incorporating additional modalities (such as image inputs) into LLMs is viewed by some as a key frontier in AI research and development.”

虽然许多人仍将 Gemini 和 GPT-4V 称为法学硕士 (LLM)，但它们更适合被称为基础模型。 “基础”一词既表明了这些模型在人工智能应用中的重要性，也表明它们可以根据不同的需求进行构建。

While many people still call Gemini and GPT-4V LLMs, they’re better characterized as foundation models. The word foundation signifies both the importance of these models in AI applications and the fact that they can be built upon for different needs.

基础模型标志着人工智能研究传统结构的突破。长期以来，人工智能研究被数据模态划分。自然语言处理 (NLP) 仅处理文本。计算机视觉仅处理视觉。纯文本模型可用于翻译和垃圾邮件检测等任务。纯图像模型可用于对象检测和图像分类。纯音频模型可以处理语音识别（语音转文本，简称 STT）和语音合成（文本转语音，简称 TTS）。

Foundation models mark a breakthrough from the traditional structure of AI research. For a long time, AI research was divided by data modalities. Natural language processing (NLP) deals only with text. Computer vision deals only with vision. Text-only models can be used for tasks such as translation and spam detection. Image-only models can be used for object detection and image classification. Audio-only models can handle speech recognition (speech-to-text, or STT) and speech synthesis (text-to-speech, or TTS).

能够处理多种数据模态的模型也称为多模态模型。生成式多模态模型也称为大型多模态模型 (LMM)。如果说语言模型以纯文本标记为条件生成下一个标记，那么多模态模型则会同时以文本和图像标记为条件，或者以模型支持的任何模态为条件生成下一个标记，如图1-3所示。

A model that can work with more than one data modality is also called a multimodal model. A generative multimodal model is also called a large multimodal model (LMM). If a language model generates the next token conditioned on text-only tokens, a multimodal model generates the next token conditioned on both text and image tokens, or whichever modalities that the model supports, as shown in Figure 1-3.

图 1-3。多模态模型可以使用来自文本和视觉标记的信息来生成下一个标记。

与语言模型一样，多模态模型也需要数据来扩展。自监督也适用于多模态模型。例如，OpenAI 使用了一种名为自然语言监督的自监督变体来训练其语言-图像模型CLIP (OpenAI, 2021)。他们没有手动为每幅图像生成标签，而是找到了互联网上同时出现的（图像，文本）对。他们能够生成一个包含 4 亿个（图像，文本）对的数据集，这个数据集比 ImageNet 大 400 倍，而且无需人工标注成本。该数据集使 CLIP 成为第一个无需额外训练即可推广到多个图像分类任务的模型。

Just like language models, multimodal models need data to scale up. Self-supervision works for multimodal models too. For example, OpenAI used a variant of self-supervision called natural language supervision to train their language-image model CLIP (OpenAI, 2021). Instead of manually generating labels for each image, they found (image, text) pairs that co-occurred on the internet. They were able to generate a dataset of 400 million (image, text) pairs, which was 400 times larger than ImageNet, without manual labeling cost. This dataset enabled CLIP to become the first model that could generalize to multiple image classification tasks without requiring additional training.

请注意，CLIP 不是一个生成模型——它没有经过训练来生成开放式输出。CLIP 是一个嵌入模型，经过训练可以生成文本和图像的联合嵌入。“嵌入简介”详细讨论了嵌入。目前，您可以将嵌入视为旨在捕捉原始数据含义的向量。像 CLIP 这样的多模态嵌入模型是生成式多模态模型（例如 Flamingo、LLaVA 和 Gemini（前身为 Bard））的支柱。

Note that CLIP isn’t a generative model—it wasn’t trained to generate open-ended outputs. CLIP is an embedding model, trained to produce joint embeddings of both texts and images. “Introduction to Embedding” discusses embeddings in detail. For now, you can think of embeddings as vectors that aim to capture the meanings of the original data. Multimodal embedding models like CLIP are the backbones of generative multimodal models, such as Flamingo, LLaVA, and Gemini (previously Bard).

基础模型也标志着从特定任务模型到通用模型的转变。以前，模型通常是为特定任务开发的，例如情绪分析或翻译。为情绪分析训练的模型无法进行翻译，反之亦然。

Foundation models also mark the transition from task-specific models to general-purpose models. Previously, models were often developed for specific tasks, such as sentiment analysis or translation. A model trained for sentiment analysis wouldn’t be able to do translation, and vice versa.

基础模型得益于其规模和训练方式，能够执行各种任务。开箱即用的通用模型在许多任务上都能表现得相对良好。LLM 可以进行情感分析和翻译。然而，你通常可以调整通用模型，以最大限度地提高其在特定任务上的性能。

Foundation models, thanks to their scale and the way they are trained, are capable of a wide range of tasks. Out of the box, general-purpose models can work relatively well for many tasks. An LLM can do both sentiment analysis and translation. However, you can often tweak a general-purpose model to maximize its performance on a specific task.

图 1-4显示了 Super-NaturalInstructions 基准用于评估基础模型的任务（Wang 等人，2022 年），从而提供了基础模型可以执行的任务类型的概念。

Figure 1-4 shows the tasks used by the Super-NaturalInstructions benchmark to evaluate foundation models (Wang et al., 2022), providing an idea of the types of tasks a foundation model can perform.

想象一下，您正在与一家零售商合作，开发一款应用程序来为其网站生成产品描述。一个现成的模型或许能够生成准确的描述，但可能无法捕捉品牌的声音或突出品牌的信息。生成的描述甚至可能充斥着营销术语和陈词滥调。

Imagine you’re working with a retailer to build an application to generate product descriptions for their website. An out-of-the-box model might be able to generate accurate descriptions but might fail to capture the brand’s voice or highlight the brand’s messaging. The generated descriptions might even be full of marketing speech and cliches.

图 1-4 Super-NaturalInstructions 基准测试中的任务范围（Wang 等人，2022 年）。

您可以使用多种技术来让模型生成您想要的内容。例如，您可以根据理想产品描述的示例，精心设计详细的说明。这种方法被称为“快速工程”。您可以将模型连接到客户评论数据库，模型可以利用该数据库生成更准确的描述。使用数据库来补充说明被称为检索增强生成(RAG)。您还可以使用高质量的产品描述数据集对模型进行微调，即进一步训练。

There are multiple techniques you can use to get the model to generate what you want. For example, you can craft detailed instructions with examples of the desirable product descriptions. This approach is prompt engineering. You can connect the model to a database of customer reviews that the model can leverage to generate better descriptions. Using a database to supplement the instructions is called retrieval-augmented generation (RAG). You can also finetune—further train—the model on a dataset of high-quality product descriptions.

快速工程、RAG 和微调是三种非常常见的 AI 工程技术，你可以使用它们来调整模型以满足你的需求。本书的其余部分将详细讨论所有这些技术。

Prompt engineering, RAG, and finetuning are three very common AI engineering techniques that you can use to adapt a model to your needs. The rest of the book will discuss all of them in detail.

Adapting an existing powerful model to your task is generally a lot easier than building a model for your task from scratch—for example, ten examples and one weekend versus 1 million examples and six months. Foundation models make it cheaper to develop AI applications and reduce time to market. Exactly how much data is needed to adapt a model depends on what technique you use. This book will also touch on this question when discussing each technique. However, there are still many benefits to task-specific models, for example, they might be a lot smaller, making them faster and cheaper to use.

Adapting an existing powerful model to your task is generally a lot easier than building a model for your task from scratch—for example, ten examples and one weekend versus 1 million examples and six months. Foundation models make it cheaper to develop AI applications and reduce time to market. Exactly how much data is needed to adapt a model depends on what technique you use. This book will also touch on this question when discussing each technique. However, there are still many benefits to task-specific models, for example, they might be a lot smaller, making them faster and cheaper to use.

Whether to build your own model or leverage an existing one is a classic buy-or-build question that teams will have to answer for themselves. Discussions throughout the book can help with that decision.

Whether to build your own model or leverage an existing one is a classic buy-or-build question that teams will have to answer for themselves. Discussions throughout the book can help with that decision.

From Foundation Models to AI Engineering

From Foundation Models to AI Engineering

AI engineering refers to the process of building applications on top of foundation models. People have been building AI applications for over a decade—a process often known as ML engineering or MLOps (short for ML operations). Why do we talk about AI engineering now?

AI engineering refers to the process of building applications on top of foundation models. People have been building AI applications for over a decade—a process often known as ML engineering or MLOps (short for ML operations). Why do we talk about AI engineering now?

If traditional ML engineering involves developing ML models, AI engineering leverages existing ones. The availability and accessibility of powerful foundation models lead to three factors that, together, create ideal conditions for the rapid growth of AI engineering as a discipline:

If traditional ML engineering involves developing ML models, AI engineering leverages existing ones. The availability and accessibility of powerful foundation models lead to three factors that, together, create ideal conditions for the rapid growth of AI engineering as a discipline:

Factor 1: General-purpose AI capabilities

Factor 1: General-purpose AI capabilities

Foundation models are powerful not just because they can do existing tasks better. They are also powerful because they can do more tasks. Applications previously thought impossible are now possible, and applications not thought of before are emerging. Even applications not thought possible today might be possible tomorrow. This makes AI more useful for more aspects of life, vastly increasing both the user base and the demand for AI applications.

Foundation models are powerful not just because they can do existing tasks better. They are also powerful because they can do more tasks. Applications previously thought impossible are now possible, and applications not thought of before are emerging. Even applications not thought possible today might be possible tomorrow. This makes AI more useful for more aspects of life, vastly increasing both the user base and the demand for AI applications.

For example, since AI can now write as well as humans, sometimes even better, AI can automate or partially automate every task that requires communication, which is pretty much everything. AI is used to write emails, respond to customer requests, and explain complex contracts. Anyone with a computer has access to tools that can instantly generate customized, high-quality images and videos to help create marketing materials, edit professional headshots, visualize art concepts, illustrate books, and so on. AI can even be used to synthesize training data, develop algorithms, and write code, all of which will help train even more powerful models in the future.

For example, since AI can now write as well as humans, sometimes even better, AI can automate or partially automate every task that requires communication, which is pretty much everything. AI is used to write emails, respond to customer requests, and explain complex contracts. Anyone with a computer has access to tools that can instantly generate customized, high-quality images and videos to help create marketing materials, edit professional headshots, visualize art concepts, illustrate books, and so on. AI can even be used to synthesize training data, develop algorithms, and write code, all of which will help train even more powerful models in the future.

Factor 2: Increased AI investments

Factor 2: Increased AI investments

ChatGPT 的成功促使风险投资家和企业对人工智能的投资大幅增加。随着人工智能应用的构建成本降低、上市速度加快，人工智能的投资回报率也变得更具吸引力。各大公司纷纷将人工智能融入其产品和流程。Scribd 应用研究高级经理 Matt Ross 告诉我，从 2022 年 4 月到 2023 年 4 月，他所负责的用例的人工智能预估成本下降了两个数量级。

The success of ChatGPT prompted a sharp increase in investments in AI, both from venture capitalists and enterprises. As AI applications become cheaper to build and faster to go to market, returns on investment for AI become more attractive. Companies rush to incorporate AI into their products and processes. Matt Ross, a senior manager of applied research at Scribd, told me that the estimated AI cost for his use cases has gone down two orders of magnitude from April 2022 to April 2023.

高盛研究估计，到2025年，美国的人工智能投资可能接近1000亿美元，全球投资可能接近2000亿美元。9^人工智能经常被视为一项竞争优势。FactSet发现，三分之一的标普500指数公司在2023年第二季度的财报电话会议上提到了人工智能，这一数字是上一年的三倍。图1-5显示了2018年至2023年期间在财报电话会议上提到人工智能的标普500指数公司的数量。

图 1-5。2023年，在财报电话会议上提及人工智能的标准普尔 500 指数公司数量创历史新高。数据来自 FactSet。

Goldman Sachs Research estimated that AI investment could approach $100 billion in the US and $200 billion globally by 2025.⁹ AI is often mentioned as a competitive advantage. FactSet found that one in three S&P 500 companies mentioned AI in their earnings calls for the second quarter of 2023, three times more than did so the year earlier. Figure 1-5 shows the number of S&P 500 companies that mentioned AI in their earning calls from 2018 to 2023.

Figure 1-5. The number of S&P 500 companies that mention AI in their earnings calls reached a record high in 2023. Data from FactSet.

根据 WallStreetZen 的数据，在财报电话会议上提及人工智能的公司，其股价涨幅高于未提及人工智能的公司：前者平均涨幅为 4.6%，后者平均涨幅为 2.4%。目前尚不清楚这其中是否存在因果关系（人工智能使这些公司更加成功）或相关性（公司之所以成功，是因为它们能够快速适应新技术）。

According to WallStreetZen, companies that mentioned AI in their earning calls saw their stock price increase more than those that didn’t: an average of a 4.6% increase compared to 2.4%. It’s unclear whether it’s causation (AI makes these companies more successful) or correlation (companies are successful because they are quick to adapt to new technologies).

因素三：构建人工智能应用的门槛低

Factor 3: Low entrance barrier to building AI applications

OpenAI 和其他模型提供商推广的“模型即服务”方法，让利用 AI 构建应用程序变得更加轻松。在这种方法中，模型通过 API 公开，这些 API 接收用户查询并返回模型输出。如果没有这些 API，使用 AI 模型就需要基础设施来托管和服务该模型。这些 API 让您只需调用一次即可访问强大的模型。

The model as a service approach popularized by OpenAI and other model providers makes it easier to leverage AI to build applications. In this approach, models are exposed via APIs that receive user queries and return model outputs. Without these APIs, using an AI model requires the infrastructure to host and serve this model. These APIs give you access to powerful models via single API calls.

不仅如此，人工智能还能以最少的编码构建应用程序。首先，人工智能可以为你编写代码，让没有软件工程背景的人也能快速将他们的想法转化为代码，并交付给用户。其次，你可以用简单的英语使用这些模型，而不必使用编程语言。现在，任何人，我的意思是任何人，都可以开发人工智能应用程序。

Not only that, AI also makes it possible to build applications with minimal coding. First, AI can write code for you, allowing people without a software engineering background to quickly turn their ideas into code and put them in front of their users. Second, you can work with these models in plain English instead of having to use a programming language. Anyone, and I mean anyone, can now develop AI applications.

由于开发基础模型所需的资源，这一过程只有大型企业（谷歌、Meta、微软、百度、腾讯）、政府（日本、阿联酋）以及雄心勃勃、资金充足的初创公司（OpenAI、Anthropic、Mistral）才有可能实现。OpenAI首席执行官 Sam Altman在 2022 年 9 月的一次采访中表示，对于绝大多数人来说，最大的机会将是将这些模型应用于特定的应用。

Because of the resources it takes to develop foundation models, this process is possible only for big corporations (Google, Meta, Microsoft, Baidu, Tencent), governments (Japan, the UAE), and ambitious, well-funded startups (OpenAI, Anthropic, Mistral). In a September 2022 interview, Sam Altman, CEO of OpenAI, said that the biggest opportunity for the vast majority of people will be to adapt these models for specific applications.

全世界都在迅速拥抱这一机遇。人工智能工程已迅速崛起，成为发展最快的工程学科之一，甚至可能是增长最快的学科。人工智能工程工具的吸引力比以往任何软件工程工具都更快。短短两年内，四款开源人工智能工程工具（AutoGPT、Stable Diffusion eb UI、LangChain、Ollama）在 GitHub 上获得的 Stars 数量已超过比特币。它们甚至有望在 Stars 数量上超越最流行的 Web 开发框架，包括 React 和 Vue。图 1-6展示了人工智能工程工具与比特币、Vue 和 React 的 GitHub Stars 增长情况对比。

The world is quick to embrace this opportunity. AI engineering has rapidly emerged as one of the fastest, and quite possibly the fastest-growing, engineering discipline. Tools for AI engineering are gaining traction faster than any previous software engineering tools. Within just two years, four open source AI engineering tools (AutoGPT, Stable Diffusion eb UI, LangChain, Ollama) have already garnered more stars on GitHub than Bitcoin. They are on track to surpass even the most popular web development frameworks, including React and Vue, in star count. Figure 1-6 shows the GitHub star growth of AI engineering tools compared to Bitcoin, Vue, and React.

LinkedIn于 2023 年 8 月开展的一项调查显示，在个人资料中添加“生成式人工智能”、“ChatGPT”、“快速工程”和“快速制作”等术语的专业人士数量平均每月增长 75%。ComputerWorld宣称，“教会人工智能如何行动是增长最快的职业技能”。。

A LinkedIn survey from August 2023 shows that the number of professionals adding terms like “Generative AI,” “ChatGPT,” “Prompt Engineering,” and “Prompt Crafting” to their profile increased on average 75% each month. ComputerWorld declared that “teaching AI to behave is the fastest-growing career skill”.

图 1-6.根据 GitHub 上的 star 数量，开源 AI 工程工具的增长速度比任何其他软件工程工具都要快。

人工智能工程师群体的快速扩张展现了非凡的创造力，并催生出一系列激动人心的应用。下一节将探讨一些最常见的应用模式。

The rapidly expanding community of AI engineers has demonstrated remarkable creativity with an incredible range of exciting applications. The next section will explore some of the most common application patterns.

编码

Coding

在多项生成式人工智能调查中，编程无疑是最受欢迎的用例。人工智能编程工具之所以受欢迎，一方面是因为人工智能本身就擅长编程，另一方面是因为早期的人工智能工程师本身就是程序员，他们更容易接触到编程挑战。

In multiple generative AI surveys, coding is hands down the most popular use case. AI coding tools are popular both because AI is good at coding and because early AI engineers are coders who are more exposed to coding challenges.

基础模型最早在生产环境中取得成功的例子之一是代码补全工具 GitHub Copilot，其年经常性收入在推出仅两年后就突破了 1 亿美元。截至本文撰写时，人工智能驱动的编程初创公司已筹集了数亿美元，其中Magic 筹集了 3.2 亿美元，Anysphere 筹集了 6000 万美元，这两家公司均在 2024 年 8 月完成融资。gpt -engineer和screen-to-code等开源编程工具在一年内都在 GitHub 上获得了 5 万颗星，并且还有更多工具正在迅速涌现。

One of the earliest successes of foundation models in production is the code completion tool GitHub Copilot, whose annual recurring revenue crossed $100 million only two years after its launch. As of this writing, AI-powered coding startups have raised hundreds of millions of dollars, with Magic raising $320 million and Anysphere raising $60 million, both in August 2024. Open source coding tools like gpt-engineer and screenshot-to-code both got 50,000 stars on GitHub within a year, and many more are being rapidly introduced.

除了帮助完成常规编码的工具外，还有许多工具专门用于某些编码任务。以下是这些任务的示例：

Other than tools that help with general coding, many tools specialize in certain coding tasks. Here are examples of these tasks:

• 从网页和 PDF 中提取结构化数据（AgentGPT）

• Extracting structured data from web pages and PDFs (AgentGPT)

• 将英语转换为代码（DB-GPT，SQL Chat，PandasAI）

• Converting English to code (DB-GPT, SQL Chat, PandasAI)

• 给定一个设计或屏幕截图，生成将呈现为类似于给定图像的网站的代码（屏幕截图到代码，绘制用户界面）

• Given a design or a screenshot, generating code that will render into a website that looks like the given image (screenshot-to-code, draw-a-ui)

• 从一种编程语言或框架转换到另一种编程语言或框架（GPT-Migrate，AI 代码翻译器）

• Translating from one programming language or framework to another (GPT-Migrate, AI Code Translator)

• 编写文档（Autodoc）

• Writing documentation (Autodoc)

• 创建测试（PentestGPT）

• Creating tests (PentestGPT)

• 生成提交消息（AI 提交）

• Generating commit messages (AI Commits)

显然，人工智能可以完成许多软件工程任务。问题在于人工智能能否完全实现软件工程自动化。一方面，NVIDIA 首席执行官黄仁勋预测人工智能将取代人类软件工程师，我们应该停止说孩子们应该学习编程。在一份泄露的录音中，AWS 首席执行官马特·加曼表示，在不久的将来，大多数开发人员将不再从事编程工作。他的意思并非软件开发人员的终结，只是他们的工作将发生变化。

It’s clear that AI can do many software engineering tasks. The question is whether AI can automate software engineering altogether. At one end of the spectrum, Jensen Huang, CEO of NVIDIA, predicts that AI will replace human software engineers and that we should stop saying kids should learn to code. In a leaked recording, AWS CEO Matt Garman shared that in the near future, most developers will stop coding. He doesn’t mean it as the end of software developers; it’s just that their jobs will change.

另一方面，许多软件工程师坚信他们永远不会被人工智能取代，既出于技术原因，也出于情感原因（人们不喜欢承认他们可以被取代）。

At the other end are many software engineers who are convinced that they will never be replaced by AI, both for technical and emotional reasons (people don’t like admitting that they can be replaced).

软件工程包含许多任务。人工智能在某些方面比其他方面更胜一筹。麦肯锡的研究人员发现，人工智能可以帮助开发人员将文档编写效率提高一倍，将代码生成和代码重构效率提高 25% 至 50%。对于高度复杂的任务，生产力提升幅度微乎其微，如图 1-9所示。在我与 AI 编程工具开发人员的交谈中，许多人告诉我，他们注意到人工智能在前端开发方面的表现远优于后端开发。

Software engineering consists of many tasks. AI is better at some than others. McKinsey researchers found that AI can help developers be twice as productive for documentation, and 25–50% more productive for code generation and code refactoring. Minimal productivity improvement was observed for highly complex tasks, as shown in Figure 1-9. In my conversations with developers of AI coding tools, many told me that they’ve noticed that AI is much better at frontend development than backend development.

图 1-9。人工智能可以帮助开发人员显著提高工作效率，尤其是在执行简单任务时，但对于高度复杂的任务，其作用则较小。数据来自麦肯锡。

无论人工智能是否会取代软件工程师，它肯定能提高他们的工作效率。这意味着公司现在可以用更少的工程师完成更多工作。人工智能还可能颠覆外包行业，因为外包任务往往是公司核心业务之外的简单任务。

Regardless of whether AI will replace software engineers, AI can certainly make them more productive. This means that companies can now accomplish more with fewer engineers. AI can also disrupt the outsourcing industry, as outsourced tasks tend to be simpler ones outside of a company’s core business.

图像和视频制作

Image and Video Production

得益于其概率特性，人工智能非常适合创造性任务。一些最成功的人工智能初创公司都是创意应用，例如用于图像生成的 Midjourney、用于照片编辑的 Adobe Firefly，以及用于视频生成的 Runway、Pika Labs 和 Sora。2023 年末，成立一年半的Midjourney已创造了 2 亿美元的年经常性收入。截至 2023 年 12 月，在 Apple App Store 上排名前 10 的免费图形与设计应用中，有一半的名称中都带有“AI”字样。我估计，图形和设计应用很快就会默认集成 AI，它们的名称中不再需要“AI”一词。第二章将更详细地讨论人工智能的概率特性。

Thanks to its probabilistic nature, AI is great for creative tasks. Some of the most successful AI startups are creative applications, such as Midjourney for image generation, Adobe Firefly for photo editing, and Runway, Pika Labs, and Sora for video generation. In late 2023, at one and a half years old, Midjourney had already generated $200 million in annual recurring revenue. As of December 2023, among the top 10 free apps for Graphics & Design on the Apple App Store, half have AI in their names. I suspect that soon, graphics and design apps will incorporate AI by default, and they’ll no longer need the word “AI” in their names. Chapter 2 discusses the probabilistic nature of AI in more detail.

如今，从领英 (LinkedIn) 到抖音 (TikTok)，使用人工智能 (AI) 生成社交媒体头像已是司空见惯。许多求职者认为，人工智能生成的头像可以帮助他们展现最佳形象，并提高求职成功率。人们对人工智能生成的头像的看法已发生显著变化。2019 年，Facebook出于安全考虑封禁了使用人工智能生成的头像的账户。2023 年，许多社交媒体应用程序将提供允许用户使用人工智能生成头像的工具。

It’s now common to use AI to generate profile pictures for social media, from LinkedIn to TikTok. Many candidates believe that AI-generated headshots can help them put their best foot forward and increase their chances of landing a job. The perception of AI-generated profile pictures has changed significantly. In 2019, Facebook banned accounts using AI-generated profile photos for safety reasons. In 2023, many social media apps provide tools that let users use AI to generate profile photos.

对于企业而言，广告和营销已迅速融入人工智能。12^人工智能可以直接用于生成宣传图片和视频。它可以帮助集思广益，或生成初稿供人类专家进行迭代。您可以使用人工智能生成多个广告，并测试哪一个最适合受众。人工智能可以根据季节和地点生成不同的广告。例如，您可以使用人工智能在秋季改变树叶颜色，或在冬季在地面上添加积雪。

For enterprises, ads and marketing have been quick to incorporate AI.¹² AI can be used to generate promotional images and videos directly. It can help brainstorm ideas or generate first drafts for human experts to iterate upon. You can use AI to generate multiple ads and test to see which one works the best for the audience. AI can generate variations of your ads according to seasons and locations. For example, you can use AI to change leaf colors during fall or add snow to the ground during winter.

写作

Writing

人工智能早已被用于辅助写作。如果你使用智能手机，你可能对自动更正和自动完成功能并不陌生，这两项功能都由人工智能驱动。写作是人工智能的理想应用，因为我们经常写作，写作过程可能相当繁琐，而且我们对错误的容忍度很高。如果模型提出了你不喜欢的建议，你可以直接忽略它。

AI has long been used to aid writing. If you use a smartphone, you’re probably familiar with autocorrect and auto-completion, both powered by AI. Writing is an ideal application for AI because we do it a lot, it can be quite tedious, and we have a high tolerance for mistakes. If a model suggests something that you don’t like, you can just ignore it.

由于法学硕士接受过文本完成方面的训练，因此他们擅长写作并不奇怪。为了研究 ChatGPT 对写作的影响，麻省理工学院的一项研究（Noy 和 Zhang，2023 年）为 453 名受过大学教育的专业人士分配了特定职业的写作任务，并随机让其中一半人接触 ChatGPT。研究结果表明，在接触 ChatGPT 的人群中，平均写作时间减少了 40%，产出质量提高了 18%。ChatGPT 有助于缩小不同工作者之间的产出质量差距，这意味着它对写作意愿较弱的工作者更有帮助。实验期间接触 ChatGPT 的工作者在实验两周后报告在实际工作中使用 ChatGPT 的可能性是实验期间的 2 倍，两个月后报告在实际工作中使用 ChatGPT 的可能性是实验期间的 1.6 倍。

It’s not a surprise that LLMs are good at writing, given that they are trained for text completion. To study the impact of ChatGPT on writing, an MIT study (Noy and Zhang, 2023) assigned occupation-specific writing tasks to 453 college-educated professionals and randomly exposed half of them to ChatGPT. Their results show that among those exposed to ChatGPT, the average time taken decreased by 40% and output quality rose by 18%. ChatGPT helps close the gap in output quality between workers, which means that it’s more helpful to those with less inclination for writing. Workers exposed to ChatGPT during the experiment were 2 times as likely to report using it in their real job two weeks after the experiment and 1.6 times as likely two months after that.

对于消费者来说，用例显而易见。许多人使用人工智能来帮助他们更好地沟通。你可以在邮件中表达愤怒，并要求人工智能将其变得愉快。你可以输入要点，然后得到完整的段落。一些人声称，如果不先让人工智能改进，他们就不会再发送重要的邮件了。

For consumers, the use cases are obvious. Many use AI to help them communicate better. You can be angry in an email and ask AI to make it pleasant. You can give it bullet points and get back complete paragraphs. Several people claimed they no longer send an important email without asking AI to improve it first.

学生们正在使用人工智能写论文。作家们正在使用人工智能写书。13^许多初创公司已经利用人工智能生成儿童、同人小说、爱情小说和奇幻小说。与传统书籍不同，人工智能生成的书籍具有互动性，因为书籍的情节可以根据读者的喜好进行调整。这意味着读者可以积极参与创作他们正在阅读的故事。儿童阅读应用程序可以识别孩子难以理解的单词，并围绕这些单词生成故事。

Students are using AI to write essays. Writers are using AI to write books.¹³ Many startups already use AI to generate children’s, fan fiction, romance, and fantasy books. Unlike traditional books, AI-generated books can be interactive, as a book’s plot can change depending on a reader’s preference. This means that readers can actively participate in creating the story they are reading. A children’s reading app identifies the words that a child has trouble with and generates stories centered around these words.

Google Docs、Notion 和 Gmail 等笔记和电子邮件应用都使用人工智能来帮助用户提高写作水平。写作助手应用Grammarly会通过微调模型，使用户的写作更加流畅、连贯、清晰。

Note-taking and email apps like Google Docs, Notion, and Gmail all use AI to help users improve their writing. Grammarly, a writing assistant app, finetunes a model to make users’ writing more fluent, coherent, and clear.

人工智能的写作能力也可能被滥用。2023年，《纽约时报》报道称，亚马逊上充斥着大量由人工智能生成的劣质旅游指南，每本指南都配有作者简介、网站和好评如潮，而所有这些都是由人工智能生成的。

AI’s ability to write can also be abused. In 2023, the New York Times reported that Amazon was flooded with shoddy AI-generated travel guidebooks, each outfitted with an author bio, a website, and rave reviews, all AI-generated.

对于企业而言，AI 写作在销售、市场营销和一般团队沟通中非常常见。许多经理告诉我，他们一直在使用 AI 来帮助他们撰写绩效报告。AI 可以帮助撰写有效的外联邮件、广告文案和产品描述。像 HubSpot 和 Salesforce 这样的客户关系管理 (CRM) 应用程序也为企业用户提供了生成网页内容和外联邮件的工具。

For enterprises, AI writing is common in sales, marketing, and general team communication. Many managers told me they’ve been using AI to help them write performance reports. AI can help craft effective cold outreach emails, ad copywriting, and product descriptions. Customer relationship management (CRM) apps like HubSpot and Salesforce also have tools for enterprise users to generate web content and outreach emails.

人工智能似乎特别擅长搜索引擎优化 (SEO)，或许是因为许多人工智能模型都是用互联网数据训练的，而互联网上充斥着经过 SEO 优化的文本。人工智能如此擅长 SEO，以至于催生了新一代内容农场。这些农场建立垃圾网站，并在其中填充人工智能生成的内容，使其在谷歌上排名靠前，从而吸引流量。然后，他们通过广告交易平台出售广告位。2023 年 6 月，NewsGuard在人工智能生成的垃圾网站上发现了来自 141 个热门品牌的近 400 条广告。其中一个垃圾网站每天生成 1200 篇文章。如果不采取措施遏制这种情况，互联网内容的未来将由人工智能生成，而且前景将相当黯淡^。14

AI seems particularly good with SEO, perhaps because many AI models are trained with data from the internet, which is populated with SEO-optimized text. AI is so good at SEO that it has enabled a new generation of content farms. These farms set up junk websites and fill them with AI-generated content to get them to rank high on Google to drive traffic to them. Then they sell advertising spots through ad exchanges. In June 2023, NewsGuard identified almost 400 ads from 141 popular brands on junk AI-generated websites. One of those junk websites produced 1,200 articles a day. Unless something is done to curtail this, the future of internet content will be AI-generated, and it’ll be pretty bleak.¹⁴

教育

Education

每当 ChatGPT 宕机时，OpenAI 的 Discord 服务器就会被学生的抱怨淹没，他们抱怨无法完成作业。包括纽约市公立学校和洛杉矶联合学区在内的多个教育委员会曾迅速禁止 ChatGPT，原因是担心学生会利用它作弊，但几个月后又改变了决定。

Whenever ChatGPT is down, OpenAI’s Discord server is flooded with students complaining about being unable to complete their homework. Several education boards, including the New York City Public Schools and the Los Angeles Unified School District, were quick to ban ChatGPT for fear of students using it for cheating, but reversed their decisions just a few months later.

学校不应该禁止人工智能，而应该将其融入其中，帮助学生更快地学习。人工智能可以总结教科书，并为每个学生生成个性化的教学计划。广告是个性化的，这一点我觉得很奇怪，因为我们知道每个人都是不同的，但教育却不是。人工智能可以帮助调整学习材料，使其适应每个学生最适合的学习形式。听觉学习者可以让人工智能大声朗读学习材料。喜欢动物的学生可以使用人工智能调整可视化效果，以展现更多动物。那些觉得阅读代码比阅读数学方程式更容易的学生，可以让人工智能将数学方程式翻译成代码。

Instead of banning AI, schools could incorporate it to help students learn faster. AI can summarize textbooks and generate personalized lecture plans for each student. I find it strange that ads are personalized because we know everyone is different, but education is not. AI can help adapt the materials to the format best suited for each student. Auditory learners can ask AI to read the materials out loud. Students who love animals can use AI to adapt visualizations to feature more animals. Those who find it easier to read code than math equations can ask AI to translate math equations into code.

人工智能对语言学习尤其有用，因为你可以让人工智能扮演不同的练习场景。Pajak和 Bicknell（Duolingo，2022）发现，在课程创建的四个阶段中，课程个性化是最能从人工智能中获益的阶段，如图1-10所示。

AI is especially helpful for language learning, as you can ask AI to roleplay different practice scenarios. Pajak and Bicknell (Duolingo, 2022) found that out of four stages of course creation, lesson personalization is the stage that can benefit the most from AI, as shown in Figure 1-10.

图 1-10。人工智能可用于多邻国课程创建的所有四个阶段，但它在个性化阶段发挥的作用最大。图片来自 Pajak 和 Bicknell（多邻国，2022 年）。

人工智能可以生成测验，包括多项选择题和开放式测验，并评估答案。人工智能可以成为辩论伙伴，因为它比普通人类更擅长就同一主题提出不同的观点。例如，可汗学院为学生提供人工智能助教，为教师提供课程助理。我见过一种创新的教学方法是，教师布置人工智能生成的作文，让学生找出并纠正错误。

AI can generate quizzes, both multiple-choice and open-ended, and evaluate the answers. AI can become a debate partner as it’s much better at presenting different views on the same topic than the average human. For example, Khan Academy offers AI-powered teaching assistants to students and course assistants to teachers. An innovative teaching method I’ve seen is that teachers assign AI-generated essays for students to find and correct mistakes.

虽然许多教育公司拥抱人工智能来打造更好的产品，但许多公司也发现自己的午餐被人工智能抢走了。例如，帮助学生完成作业的公司 Chegg 的股价从 2022 年 11 月 ChatGPT 推出时的 28 美元暴跌至 2024 年 9 月的 2 美元，因为学生们纷纷向人工智能寻求帮助。

While many education companies embrace AI to build better products, many find their lunches taken by AI. For example, Chegg, a company that helps students with their homework, saw its share price plummet from $28 when ChatGPT launched in November 2022 to $2 in September 2024, as students have been turning to AI for help.

如果说风险在于人工智能可以取代许多技能，那么机遇则在于人工智能可以充当导师，帮助人们学习任何技能。对于许多技能，人工智能可以帮助人们快速上手，然后继续自主学习，最终超越人工智能。

If the risk is that AI can replace many skills, the opportunity is that AI can be used as a tutor to learn any skill. For many skills, AI can help someone get up to speed quickly and then continue learning on their own to become better than AI.

信息聚合

Information Aggregation

许多人认为，我们的成功取决于我们过滤和消化有用信息的能力。然而，处理电子邮件、Slack 消息和新闻有时会让人不知所措。幸运的是，人工智能拯救了我们。事实证明，人工智能能够聚合信息并进行总结。根据Salesforce 的《2023 年生成式人工智能快照研究》，74% 的生成式人工智能用户使用它来提炼复杂的想法并总结信息。

Many people believe that our success depends on our ability to filter and digest useful information. However, keeping up with emails, Slack messages, and news can sometimes be overwhelming. Luckily, AI came to the rescue. AI has proven to be capable of aggregating information and summarizing it. According to Salesforce’s 2023 Generative AI Snapshot Research, 74% of generative AI users use it to distill complex ideas and summarize information.

对于消费者来说，许多应用程序可以处理你的文档——合同、披露文件、论文——并允许你以对话的方式检索信息。这种用例也称为“与你的文档对话”。人工智能可以帮助你总结网站、研究成果，并根据你选择的主题创建报告。在撰写本书的过程中，我发现人工智能在总结和比较论文方面很有帮助。

For consumers, many applications can process your documents—contracts, disclosures, papers—and let you retrieve information in a conversational manner. This use case is also called talk-to-your-docs. AI can help you summarize websites, research, and create reports on the topics of your choice. During the process of writing this book, I found AI helpful for summarizing and comparing papers.

信息聚合和提炼对于企业运营至关重要。更高效的信息聚合和分散可以帮助组织精简，因为它可以减轻中层管理人员的负担。Instacart 在推出内部快件市场时发现，最受欢迎的快件模板之一是“快速分解”。该模板要求人工智能将会议记录、电子邮件和 Slack 对话汇总为事实、开放式问题和行动项目。然后，这些行动项目可以自动插入项目跟踪工具，并分配给合适的负责人。

Information aggregation and distillation are essential for enterprise operations. More efficient information aggregation and dissimilation can help an organization become leaner, as it reduces the burden on middle management. When Instacart launched an internal prompt marketplace, it discovered that one of the most popular prompt templates is “Fast Breakdown”. This template asks AI to summarize meeting notes, emails, and Slack conversations with facts, open questions, and action items. These action items can then be automatically inserted into a project tracking tool and assigned to the right owners.

人工智能可以帮助您发现有关潜在客户的关键信息并对您的竞争对手进行分析。

AI can help you surface the critical information about your potential customers and run analyses on your competitors.

收集的信息越多，整理起来就越重要。信息聚合与数据组织密不可分。

The more information you gather, the more important it is to organize it. Information aggregation goes hand in hand with data organization.

数据组织

Data Organization

关于未来，有一件事是肯定的：我们将继续产生越来越多的数据。智能手机用户将继续拍摄照片和视频。公司将继续记录有关其产品、员工和客户的一切信息。每年都会有数十亿份合同被创建。照片、视频、日志和 PDF 都是非结构化或半结构化数据。将所有这些数据组织成便于日后搜索的方式至关重要。

One thing certain about the future is that we’ll continue producing more and more data. Smartphone users will continue taking photos and videos. Companies will continue to log everything about their products, employees, and customers. Billions of contracts are being created each year. Photos, videos, logs, and PDFs are all unstructured or semistructured data. It’s essential to organize all this data in a way that can be searched later.

人工智能正好可以帮上忙。人工智能可以自动生成图片和视频的文字描述，或者帮助匹配文本查询和与之匹配的视觉效果。像 Google Photos 这样的服务已经在使用人工智能来展示符合搜索查询的图片。17 ^Google图片搜索更进一步：如果没有符合用户需求的现有图片，它可以生成一些。

AI can help with exactly that. AI can automatically generate text descriptions about images and videos, or help match text queries with visuals that match those queries. Services like Google Photos are already using AI to surface images that match search queries.¹⁷ Google Image Search goes a step further: if there’s no existing image matching users’ needs, it can generate some.

人工智能非常擅长数据分析。它可以编写程序来生成数据可视化，识别异常值，并做出收入预测等预测^。18

AI is very good with data analysis. It can write programs to generate data visualization, identify outliers, and make predictions like revenue forecasts.¹⁸

企业可以利用人工智能从非结构化数据中提取结构化信息，用于组织数据并辅助搜索。简单的用例包括自动从信用卡、驾照、收据、票据中提取信息，以及从电子邮件页脚中提取联系信息等。更复杂的用例包括从合同、报告、图表等中提取数据。预计到 2030 年，智能数据处理 (IDP) 行业规模将达到 128.1 亿美元，年增长率为 32.9%。

Enterprises can use AI to extract structured information from unstructured data, which can be used to organize data and help search it. Simple use cases include automatically extracting information from credit cards, driver’s licenses, receipts, tickets, contact information from email footers, and so on. More complex use cases include extracting data from contracts, reports, charts, and more. It’s estimated that the IDP, intelligent data processing, industry will reach $12.81 billion by 2030, growing 32.9% each year.

工作流自动化

Workflow Automation

最终，人工智能应该尽可能地实现自动化。对于最终用户来说，自动化可以帮助他们完成一些枯燥的日常任务，例如预订餐厅、申请退款、规划行程和填写表格。

Ultimately, AI should automate as much as possible. For end users, automation can help with boring daily tasks like booking restaurants, requesting refunds, planning trips, and filling out forms.

对于企业而言，AI 可以自动化执行重复性任务，例如潜在客户管理、开票、报销、管理客户请求、数据录入等等。一个尤其令人兴奋的用例是使用 AI 模型来合成数据，然后利用这些数据来改进模型本身。您可以使用 AI 为数据创建标签，并引入人工来改进标签。我们将在第 8 章讨论数据合成。

For enterprises, AI can automate repetitive tasks such as lead management, invoicing, reimbursements, managing customer requests, data entry, and so on. One especially exciting use case is using AI models to synthesize data, which can then be used to improve the models themselves. You can use AI to create labels for your data, looping in humans to improve the labels. We discuss data synthesis in Chapter 8.

许多任务都需要访问外部工具。例如，要预订餐厅，应用程序可能需要获得权限才能打开搜索引擎查找餐厅号码、使用手机拨打电话以及将预约添加到日历。能够规划和使用工具的人工智能被称为代理。人们对代理的兴趣近乎痴迷，但这并非完全没有道理。人工智能代理有潜力让每个人的工作效率显著提高，并创造更大的经济价值。代理是第六章的核心主题。

Access to external tools is required to accomplish many tasks. To book a restaurant, an application might need permission to open a search engine to look up the restaurant’s number, use your phone to make calls, and add appointments to your calendar. AIs that can plan and use tools are called agents. The level of interest around agents borders on obsession, but it’s not entirely unwarranted. AI agents have the potential to make every person vastly more productive and generate vastly more economic value. Agents are a central topic in Chapter 6.

研究不同的 AI 应用非常有趣。我最喜欢幻想的事情之一就是能够构建各种不同的应用。然而，并非所有应用都应该构建。下一节将讨论在构建 AI 应用之前我们应该考虑哪些因素。

It’s been a lot of fun looking into different AI applications. One of my favorite things to daydream about is the different applications I can build. However, not all applications should be built. The next section discusses what we should consider before building an AI application.

用例评估

Use Case Evaluation

首先要问的问题是，你为什么要构建这个应用程序。与许多商业决策一样，构建人工智能应用程序通常是对风险和机遇的回应。以下是一些不同风险级别的示例，按风险等级从高到低排序：

The first question to ask is why you want to build this application. Like many business decisions, building an AI application is often a response to risks and opportunities. Here are a few examples of different levels of risks, ordered from high to low:

1. 如果你不这样做，拥有 AI 的竞争对手可能会让你被淘汰。如果 AI 对你的企业构成重大的生存威胁，那么采用 AI 必须是当务之急。在 2023 年Gartner 的研究中，7% 的人表示业务连续性是他们采用 AI 的原因。这在涉及文档处理和信息聚合的业务中更为常见，例如财务分析、保险和数据处理。这在广告、网页设计和图像制作等创意工作中也很常见。你可以参考 2023 年 OpenAI 的研究“GPT 就是 GPT”（Eloundou 等人，2023 年），了解各行业在 AI 暴露程度方面的排名。

2. If you don’t do this, competitors with AI can make you obsolete. If AI poses a major existential threat to your business, incorporating AI must have the highest priority. In the 2023 Gartner study, 7% cited business continuity as their reason for embracing AI. This is more common for businesses involving document processing and information aggregation, such as financial analysis, insurance, and data processing. This is also common for creative work such as advertising, web design, and image production. You can refer to the 2023 OpenAI study, “GPTs are GPTs” (Eloundou et al., 2023), to see how industries rank in their exposure to AI.

3. 如果不这样做，您将错失提升利润和生产力的机会。大多数公司都拥抱人工智能，因为它带来了诸多机遇。人工智能可以助力大多数（即使不是全部）业务运营。人工智能可以通过打造更有效的文案、产品描述和促销视觉内容，降低用户获取成本。人工智能可以通过改进客户支持和定制用户体验来提高用户留存率。人工智能还可以帮助销售线索生成、内部沟通、市场调研和竞争对手追踪。

4. If you don’t do this, you’ll miss opportunities to boost profits and productivity. Most companies embrace AI for the opportunities it brings. AI can help in most, if not all, business operations. AI can make user acquisition cheaper by crafting more effective copywrites, product descriptions, and promotional visual content. AI can increase user retention by improving customer support and customizing user experience. AI can also help with sales lead generation, internal communication, market research, and competitor tracking.

5. 您尚不确定人工智能将如何融入您的业务，但您不想落后。虽然公司不应该追逐每一次炒作，但许多公司因为等待太久而失败（例如柯达、百视达和黑莓）。如果您负担得起，投入资源去了解一项新的、变革性的技术将如何影响您的业务并非坏事。在规模较大的公司，这可以成为研发部门的一部分^。19

6. You’re unsure where AI will fit into your business yet, but you don’t want to be left behind. While a company shouldn’t chase every hype train, many have failed by waiting too long to take the leap (cue Kodak, Blockbuster, and BlackBerry). Investing resources into understanding how a new, transformational technology can impact your business isn’t a bad idea if you can afford it. At bigger companies, this can be part of the R&D department.¹⁹

一旦找到了开发此用例的充分理由，您可能会考虑是否必须自行构建。如果人工智能对您的业务构成生存威胁，您可能希望内部开发人工智能，而不是将其外包给竞争对手。但是，如果您使用人工智能来提高利润和生产力，您可能有很多购买选择，可以节省您的时间和金钱，同时提高您的绩效。

Once you’ve found a good reason to develop this use case, you might consider whether you have to build it yourself. If AI poses an existential threat to your business, you might want to do AI in-house instead of outsourcing it to a competitor. However, if you’re using AI to boost profits and productivity, you might have plenty of buy options that can save you time and money while giving you better performance.

设定期望

Setting Expectations

一旦你决定要自己构建这个令人惊叹的AI应用程序，下一步就是弄清楚成功是什么样子：你如何衡量成功？最重要的指标是它将如何影响你的业务。例如，如果它是一个客户支持聊天机器人，那么业务指标可以包括以下内容：

Once you’ve decided that you need to build this amazing AI application by yourself, the next step is to figure out what success looks like: how will you measure success? The most important metric is how this will impact your business. For example, if it’s a customer support chatbot, the business metrics can include the following:

• 您希望聊天机器人自动处理多少比例的客户消息？

• What percentage of customer messages do you want the chatbot to automate?

• 聊天机器人还应该允许您处理多少条消息？

• How many more messages should the chatbot allow you to process?

• 使用聊天机器人您可以多快做出回应？

• How much quicker can you respond using the chatbot?

• 聊天机器人可以为您节省多少人力？

• How much human labor can the chatbot save you?

聊天机器人可以回复更多消息，但这并不意味着它就能让用户满意，因此跟踪客户满意度和客户反馈非常重要。“用户反馈”探讨了如何设计反馈系统。

A chatbot can answer more messages, but that doesn’t mean it’ll make users happy, so it’s important to track customer satisfaction and customer feedback in general. “User Feedback” discusses how to design a feedback system.

为了确保产品在尚未准备好之前就交付给客户，需要对其实用性阈值设定明确的预期：产品要达到什么程度才算有用。实用性阈值可能包含以下指标：

To ensure a product isn’t put in front of customers before it’s ready, have clear expectations on its usefulness threshold: how good it has to be for it to be useful. Usefulness thresholds might include the following metrics groups:

• 用于衡量聊天机器人响应质量的质量指标。

• Quality metrics to measure the quality of the chatbot’s responses.

• 延迟指标包括 TTFT（第一个令牌时间）、TPOT（每个输出令牌时间）和总延迟。可接受的延迟取决于您的用例。如果您所有客户请求目前都由人工处理，平均响应时间为一小时，那么任何比这更快的延迟可能就足够了。

• Latency metrics including TTFT (time to first token), TPOT (time per output token), and total latency. What is considered acceptable latency depends on your use case. If all of your customer requests are currently being processed by humans with a median response time of an hour, anything faster than this might be good enough.

• 成本指标：每个推理请求的成本是多少。

• Cost metrics: how much it costs per inference request.

• 其他指标，如可解释性和公平性。

• Other metrics such as interpretability and fairness.

如果您还不确定要使用哪些指标，请不要担心。本书的其余部分将涵盖其中的许多指标。

If you’re not yet sure what metrics you want to use, don’t worry. The rest of the book will cover many of these metrics.

里程碑规划

Milestone Planning

设定可衡量的目标后，您需要制定一个计划来实现这些目标。如何实现目标取决于您的起点。评估现有模型以了解其功能。现成的模型越强大，您需要做的工作就越少。例如，如果您的目标是实现 60% 的客户支持工单的自动化，而您想要使用的现成模型已经可以实现 30% 的工单自动化，那么您需要投入的工作量可能比完全无法实现工单自动化的情况要少。

Once you’ve set measurable goals, you need a plan to achieve these goals. How to get to the goals depends on where you start. Evaluate existing models to understand their capabilities. The stronger the off-the-shelf models, the less work you’ll have to do. For example, if your goal is to automate 60% of customer support tickets and the off-the-shelf model you want to use can already automate 30% of the tickets, the effort you need to put in might be less than if it can automate no tickets at all.

评估后，你的目标很可能会发生变化。例如，评估后，你可能会意识到，让应用达到实用性阈值所需的资源将超过其潜在回报，因此你不再想继续使用它。

It’s likely that your goals will change after evaluation. For example, after evaluation, you may realize that the resources needed to get the app to the usefulness threshold will be more than its potential return, and, therefore, you no longer want to pursue it.

规划 AI 产品时需要考虑“最后一公里”的挑战。基础模型的初步成功可能会产生误导。由于基础模型的基本功能已经相当出色，构建一个有趣的演示可能并不需要花费太多时间。然而，一个好的初始演示并不能保证最终产品的质量。构建一个演示可能只需要一个周末，但打造一个产品却需要数月甚至数年的时间。

Planning an AI product needs to account for its last mile challenge. Initial success with foundation models can be misleading. As the base capabilities of foundation models are already quite impressive, it might not take much time to build a fun demo. However, a good initial demo doesn’t promise a good end product. It might take a weekend to build a demo but months, and even years, to build a product.

在论文《UltraChat》中，Ding 等人 (2023)指出：“从 0 到 60 的旅程很容易，而从 60 到 100 则极具挑战性。” LinkedIn (2024) 也表达了同样的观点。他们花了一个月的时间才达到预期体验的 80%。最初的成功让他们严重低估了改进产品所需的时间。他们发现，又花了四个月的时间才最终超过 95%。他们花费了大量时间来解决产品问题和处理各种幻觉。后续每次 1% 的提升速度都如此缓慢，令人沮丧。

In the paper UltraChat, Ding et al. (2023) shared that “the journey from 0 to 60 is easy, whereas progressing from 60 to 100 becomes exceedingly challenging.” LinkedIn (2024) shared the same sentiment. It took them one month to achieve 80% of the experience they wanted. This initial success made them grossly underestimate how much time it’d take them to improve the product. They found it took them four more months to finally surpass 95%. A lot of time was spent working on the product kinks and dealing with hallucinations. The slow speed of achieving each subsequent 1% gain was discouraging.

维护

Maintenance

产品规划并不止于实现其目标。你需要考虑产品会如何随时间变化，以及如何维护。人工智能产品的维护面临着人工智能快速变化带来的额外挑战。过去十年，人工智能领域发展迅猛，未来十年可能仍将持续快速发展。如今，在基础模型之上构建产品意味着要全力以赴地搭乘这趟高速列车。

Product planning doesn’t stop at achieving its goals. You need to think about how this product might change over time and how it should be maintained. Maintenance of an AI product has the added challenge of AI’s fast pace of change. The AI space has been moving incredibly fast in the last decade. It’ll probably continue moving fast for the next decade. Building on top of foundation models today means committing to riding this bullet train.

许多变化都是好的。例如，许多模型的局限性正在得到解决。上下文长度越来越长。模型输出越来越好。模型推理（即根据输入计算输出的过程）正变得越来越快，成本也越来越低。图 1-11展示了 2022 年至 2024 年间，在流行的基础模型基准——大规模多任务语言理解 (MMLU)（Hendrycks 等人，2020 年）上，推理成本和模型性能的演变情况。

Many changes are good. For example, the limitations of many models are being addressed. Context lengths are getting longer. Model outputs are getting better. Model inference, the process of computing an output given an input, is getting faster and cheaper. Figure 1-11 shows the evolution of inference cost and model performance on Massive Multitask Language Understanding (MMLU) (Hendrycks et al., 2020), a popular foundation model benchmark, between 2022 and 2024.

图 1-11。人工智能推理的成本随时间推移迅速下降。图片来自Katrina Nguyen（2024 年）。

然而，即使是这些好的改变也可能会给你的工作流程带来阻力。你必须时刻保持警惕，并对每项技术投资进行成本效益分析。今天最好的选择，明天可能就变成了最糟糕的选择。你可能决定自己构建模型，因为这看起来比支付模型供应商的费用更便宜，但三个月后却发现，模型供应商的价格已经降了一半，使得内部构建的成本更高。你可能会投资第三方解决方案，并围绕它定制你的基础设施，但最终供应商可能会因为无法获得资金而倒闭。

However, even these good changes can cause friction in your workflows. You’ll have to constantly be on your guard and run a cost-benefit analysis of each technology investment. The best option today might turn into the worst option tomorrow. You may decide to build a model in-house because it seems cheaper than paying for model providers, only to find out after three months that model providers have dropped their prices in half, making in-house the expensive option. You might invest in a third-party solution and tailor your infrastructure around it, only for the provider to go out of business after failing to secure funding.

有些变化更容易适应。例如，随着模型提供商逐渐采用相同的 API，将一个模型 API 替换为另一个模型 API 变得越来越容易。然而，由于每个模型都有其特性、优缺点，使用新模型的开发者需要根据新模型调整其工作流程、提示和数据。如果没有适当的版本控制和评估基础设施，这个过程可能会带来很多麻烦。

Some changes are easier to adapt to. For example, as model providers converge to the same API, it’s becoming easier to swap one model API for another. However, as each model has its quirks, strengths, and weaknesses, developers working with the new model will need to adjust their workflows, prompts, and data to this new model. Without proper infrastructure for versioning and evaluation in place, the process can cause a lot of headaches.

有些变化更难适应，尤其是法规方面的变化。许多国家将人工智能相关技术视为国家安全问题，这意味着人工智能资源（包括计算、人才和数据）受到严格监管。例如，据估计，欧洲《通用数据保护条例》(GDPR) 的出台将使企业花费90 亿美元才能达到合规要求。由于新法律对计算资源的买卖者施加了更多限制（参见美国 2023 年 10 月的行政命令），计算可用性可能会在一夜之间发生变化。如果您的 GPU 供应商突然被禁止向贵国销售 GPU，您就麻烦了。

Some changes are harder to adapt to, especially those around regulations. Technologies surrounding AI are considered national security issues for many countries, meaning resources for AI, including compute, talent, and data, are heavily regulated. The introduction of Europe’s General Data Protection Regulation (GDPR), for example, was estimated to cost businesses $9 billion to become compliant. Compute availability can change overnight as new laws put more restrictions on who can buy and sell compute resources (see the US October 2023 Executive Order). If your GPU vendor is suddenly banned from selling GPUs to your country, you’re in trouble.

有些变化甚至可能是致命的。例如，围绕知识产权 (IP) 和人工智能应用的法规仍在不断发展。如果你的产品是基于使用他人数据训练的模型构建的，你能确定产品的 IP 永远属于你吗？我采访过的许多 IP 密集型公司，例如游戏工作室，都不愿使用人工智能，因为他们担心日后会失去 IP。

Some changes can even be fatal. For example, regulations around intellectual property (IP) and AI usage are still evolving. If you build your product on top of a model trained using other people’s data, can you be certain that your product’s IP will always belong to you? Many IP-heavy companies I’ve talked to, such as game studios, hesitate to use AI for fear of losing their IPs later on.

一旦您致力于构建 AI 产品，让我们研究构建这些应用程序所需的工程堆栈。

Once you’ve committed to building an AI product, let’s look into the engineering stack needed to build these applications.

人工智能堆栈的三个层级

Three Layers of the AI Stack

任何 AI 应用程序堆栈都包含三个层级：应用程序开发、模型开发和基础架构。开发 AI 应用程序时，您通常会从顶层开始，然后根据需要向下移动：

There are three layers to any AI application stack: application development, model development, and infrastructure. When developing an AI application, you’ll likely start from the top layer and move down as needed:

应用程序开发

Application development

由于模型唾手可得，任何人都可以用它们来开发应用程序。这是过去两年中最为活跃的层面，并且仍在快速发展。应用程序开发需要为模型提供良好的提示和必要的上下文。这一层面需要严格的评估。优秀的应用程序也需要良好的界面。

With models readily available, anyone can use them to develop applications. This is the layer that has seen the most action in the last two years, and it is still rapidly evolving. Application development involves providing a model with good prompts and necessary context. This layer requires rigorous evaluation. Good applications also demand good interfaces.

模型开发

Model development

这一层提供模型开发工具，包括建模、训练、微调和推理优化的框架。由于数据是模型开发的核心，因此这一层还包含数据集工程。模型开发也需要严格的评估。

This layer provides tooling for developing models, including frameworks for modeling, training, finetuning, and inference optimization. Because data is central to model development, this layer also contains dataset engineering. Model development also requires rigorous evaluation.

基础设施

Infrastructure

堆栈底部是基础设施，其中包括用于模型服务、管理数据和计算以及监控的工具。

At the bottom is the stack is infrastructure, which includes tooling for model serving, managing data and compute, and monitoring.

这三个层以及每层的职责示例如图1-14所示。

These three layers and examples of responsibilities for each layer are shown in Figure 1-14.

图 1-14。AI工程堆栈的三个层级。

为了了解基础模型的现状演变，我在 2024 年 3 月在 GitHub 上搜索了所有至少获得 500 星的 AI 相关代码库。鉴于 GitHub 的普及性，我相信这些数据可以很好地帮助我们了解整个生态系统。在我的分析中，我还包含了应用程序和模型的代码库，它们分别是应用程序开发层和模型开发层的产物。我总共找到了 920 个代码库。图 1-15显示了每个类别代码库的月度累计数量。

To get a sense of how the landscape has evolved with foundation models, in March 2024, I searched GitHub for all AI-related repositories with at least 500 stars. Given the prevalence of GitHub, I believe this data is a good proxy for understanding the ecosystem. In my analysis, I also included repositories for applications and models, which are the products of the application development and model development layers, respectively. I found a total of 920 repositories. Figure 1-15 shows the cumulative number of repositories in each category month-over-month.

图 1-15。随时间推移按类别划分的存储库累计数量。

数据显示，在 Stable Diffusion 和 ChatGPT 推出后，2023 年 AI 工具的数量大幅增加。2023 年，增幅最高的类别是应用程序和应用程序开发。基础设施层有所增长，但增幅远低于其他层。这是意料之中的。尽管模型和应用程序发生了变化，但核心基础设施需求（资源管理、服务、监控等）保持不变。

The data shows a big jump in the number of AI toolings in 2023, after the introduction of Stable Diffusion and ChatGPT. In 2023, the categories that saw the highest increases were applications and application development. The infrastructure layer saw some growth, but it was much less than the growth seen in other layers. This is expected. Even though models and applications have changed, the core infrastructural needs—resource management, serving, monitoring, etc.—remain the same.

这将我们带到了下一点。虽然围绕基础模型的兴奋和创造力水平是前所未有的，但构建人工智能应用程序的许多原则保持不变。对于企业用例，人工智能应用程序仍然需要解决业务问题，因此，从业务指标映射到机器学习指标以及反之亦然仍然至关重要。你仍然需要进行系统性实验。使用经典的机器学习工程，你可以尝试不同的超参数。使用基础模型，你可以尝试不同的模型、提示、检索算法、采样变量等等。（采样变量将在第 2 章中讨论。）我们仍然希望让模型运行得更快、更便宜。建立反馈循环仍然很重要，这样我们就可以利用生产数据迭代地改进我们的应用程序。

This brings us to the next point. While the level of excitement and creativity around foundation models is unprecedented, many principles of building AI applications remain the same. For enterprise use cases, AI applications still need to solve business problems, and, therefore, it’s still essential to map from business metrics to ML metrics and vice versa. You still need to do systematic experimentation. With classical ML engineering, you experiment with different hyperparameters. With foundation models, you experiment with different models, prompts, retrieval algorithms, sampling variables, and more. (Sampling variables are discussed in Chapter 2.) We still want to make models run faster and cheaper. It’s still important to set up a feedback loop so that we can iteratively improve our applications with production data.

这意味着机器学习工程师在过去十年中学习和分享的许多知识仍然适用。这种集体经验使每个人都能更轻松地开始构建人工智能应用程序。然而，在这些经久不衰的原则之上，还有许多人工智能工程独有的创新，我们将在本书中进行探讨。

This means that much of what ML engineers have learned and shared over the last decade is still applicable. This collective experience makes it easier for everyone to begin building AI applications. However, built on top of these enduring principles are many innovations unique to AI engineering, which we’ll explore in this book.

人工智能工程与机器学习工程

AI Engineering Versus ML Engineering

虽然部署AI应用的不变原则令人安心，但了解情况如何变化也至关重要。这对于希望调整现有平台以适应新AI用例的团队，以及对学习哪些技能才能在新市场保持竞争力感兴趣的开发者来说，都将大有裨益。

While the unchanging principles of deploying AI applications are reassuring, it’s also important to understand how things have changed. This is helpful for teams that want to adapt their existing platforms for new AI use cases and developers who are interested in which skills to learn to stay competitive in a new market.

从高层次来看，如今使用基础模型构建应用程序与传统机器学习工程主要有以下三个不同之处：

At a high level, building applications using foundation models today differs from traditional ML engineering in three major ways:

1. 如果没有基础模型，你就必须为你的应用训练自己的模型。而有了人工智能工程，你就可以使用别人为你训练好的模型。这意味着人工智能工程的重点不再是建模和训练，而是模型自适应。

2. Without foundation models, you have to train your own models for your applications. With AI engineering, you use a model someone else has trained for you. This means that AI engineering focuses less on modeling and training, and more on model adaptation.

3. 与传统的机器学习工程相比，AI 工程需要处理的模型更大，消耗更多的计算资源，并且延迟也更高。这意味着高效训练和推理优化的压力更大。计算密集型模型的必然结果是，许多公司现在需要比以前更多的 GPU 并使用更大的计算集群，这意味着对了解如何使用 GPU 和大型集群的工程师的需求会更大^。23

4. AI engineering works with models that are bigger, consume more compute resources, and incur higher latency than traditional ML engineering. This means that there’s more pressure for efficient training and inference optimization. A corollary of compute-intensive models is that many companies now need more GPUs and work with bigger compute clusters than they previously did, which means there’s more need for engineers who know how to work with GPUs and big clusters.²³

5. 人工智能工程需要能够产生开放式输出的模型。开放式输出赋予模型执行更多任务的灵活性，但也使其更难评估。这使得评估成为人工智能工程中一个更大的难题。

6. AI engineering works with models that can produce open-ended outputs. Open-ended outputs give models the flexibility to be used for more tasks, but they are also harder to evaluate. This makes evaluation a much bigger problem in AI engineering.

简而言之，AI 工程与 ML 工程的区别在于，AI 工程关注的不是模型开发，而是模型的调整和评估。我在本章中多次提到了模型自适应，因此在继续之前，我想确保大家对模型自适应的含义达成共识。通常，模型自适应技术可以分为两类，具体取决于它们是否需要更新模型权重。

In short, AI engineering differs from ML engineering in that it’s less about model development and more about adapting and evaluating models. I’ve mentioned model adaptation several times in this chapter, so before we move on, I want to make sure that we’re on the same page about what model adaptation means. In general, model adaptation techniques can be divided into two categories, depending on whether they require updating model weights.

基于提示的技术（包括快速工程）可以在不更新模型权重的情况下调整模型。您可以通过提供指令和上下文来调整模型，而不是更改模型本身。快速工程更容易上手，并且所需的数据更少。许多成功的应用程序都是通过快速工程构建的。它的易用性使您可以尝试更多模型，从而增加了找到出乎意料地适合您应用程序的模型的机会。但是，对于复杂任务或对性能有严格要求的应用程序而言，快速工程可能不够用。

Prompt-based techniques, which include prompt engineering, adapt a model without updating the model weights. You adapt a model by giving it instructions and context instead of changing the model itself. Prompt engineering is easier to get started and requires less data. Many successful applications have been built with just prompt engineering. Its ease of use allows you to experiment with more models, which increases your chance of finding a model that is unexpectedly good for your applications. However, prompt engineering might not be enough for complex tasks or applications with strict performance requirements.

另一方面，微调需要更新模型权重。您可以通过更改模型本身来调整模型。通常，微调技术更为复杂，需要更多数据，但它可以显著改善模型的质量、延迟和成本。如果不更改模型权重，许多事情就无法实现，例如让模型适应训练期间未曾接触过的新任务。

Finetuning, on the other hand, requires updating model weights. You adapt a model by making changes to the model itself. In general, finetuning techniques are more complicated and require more data, but they can improve your model’s quality, latency, and cost significantly. Many things aren’t possible without changing model weights, such as adapting the model to a new task it wasn’t exposed to during training.

现在，让我们深入探讨应用程序开发和模型开发这两个层面，看看它们是如何随着人工智能工程的发展而发生变化的，首先从现有的机器学习工程师更熟悉的方面开始。本节概述了开发人工智能应用程序所涉及的不同流程。本书将通篇讨论这些流程的工作原理。

Now, let’s zoom into the application development and model development layers to see how each has changed with AI engineering, starting with what existing ML engineers are more familiar with. This section gives an overview of different processes involved in developing an AI application. How these processes work will be discussed throughout this book.

人工智能工程与全栈工程

AI Engineering Versus Full-Stack Engineering

对应用程序开发（尤其是接口）的日益重视，使 AI 工程更接近全栈开发。27接口重要性的日益提升促使 AI 工具的设计发生转变，以吸引更多前端工程师。传统上，机器学习工程以 Python 为中心。在基础模型出现之前，最^流行的机器学习框架主要支持 Python API。如今，Python 仍然很流行，但对 JavaScript API 的支持也日益增多，例如LangChain.js、Transformers.js、OpenAI 的 Node 库和Vercel 的 AI SDK。

The increased emphasis on application development, especially on interfaces, brings AI engineering closer to full-stack development.²⁷ The rising importance of interfaces leads to a shift in the design of AI toolings to attract more frontend engineers. Traditionally, ML engineering is Python-centric. Before foundation models, the most popular ML frameworks supported mostly Python APIs. Today, Python is still popular, but there is also increasing support for JavaScript APIs, with LangChain.js, Transformers.js, OpenAI’s Node library, and Vercel’s AI SDK.

虽然许多 AI 工程师拥有传统的机器学习背景，但越来越多的人拥有 Web 开发或全栈开发背景。全栈工程师相较于传统机器学习工程师的优势在于，他们能够快速将想法转化为演示、获得反馈并不断迭代。

While many AI engineers come from traditional ML backgrounds, more are increasingly coming from web development or full-stack backgrounds. An advantage that full-stack engineers have over traditional ML engineers is their ability to quickly turn ideas into demos, get feedback, and iterate.

在传统的机器学习工程中，通常从收集数据和训练模型开始，最后才是构建产品。然而，如今 AI 模型唾手可得，我们可以先从构建产品开始，等到产品展现出潜力后再投资数据和模型，如图1-16所示。

With traditional ML engineering, you usually start with gathering data and training a model. Building the product comes last. However, with AI models readily available today, it’s possible to start with building the product first, and only invest in data and models once the product shows promise, as visualized in Figure 1-16.

图 1-16。新的 AI 工程工作流程奖励那些能够快速迭代的人。图片来自“AI 工程师的崛起”（Shawn Wang，2023 年）。

在传统的机器学习工程中，模型开发和产品开发通常是脱节的，许多组织中的机器学习工程师很少参与产品决策。然而，在基础模型方面，人工智能工程师往往会更多地参与产品构建。

In traditional ML engineering, model development and product development are often disjointed processes, with ML engineers rarely involved in product decisions at many organizations. However, with foundation models, AI engineers tend to be much more involved in building the product.